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DETAILED ACTION 
EXAMINER'S AMENDMENT 

An examiner's amendment to the record appears below. Should the changes 
and/or additions be unacceptable to applicant, an amendment may be filed as provided 
by 37 CFR 1.312. To ensure consideration of such an amendment, it MUST be 
submitted no later than the payment of the issue fee. 

Authorization for this examiner's amendment was given in a telephone interview 
with James Singer (45,1 1 1) on Friday, January 20, 2006. 

In the claims: 

Please amend claims 1,18, 22, 26, and 30 as follows. 
SEE ATTACHED. 

REASONS FOR ALLOWANCE 

The following is an examiner's statement of reasons for allowance: 
The prior art does not particularly teach the use of a view model created with 
PTZF information in conjunction with the lack of use of image content information. 
Kreitman, Kennedy JR et al (US 2001/0031067)) teach the use of optical features to find 
various features and insert advertising. Kreitman tracks optical features as discussed in 
the previous office action. Moengen and Honey teach inserting and overlaying optical 
content in a scene without reference to three-dimensional location or position, etc. 

Any comments considered necessary by applicant must be submitted no later 
than the payment of the issue fee and, to avoid processing delays, should preferably 
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accompany the issue fee. Such submissions should be clearly labeled "Comments on 
Statement of Reasons for Allowance." 

NOTE: James Singer specifically stated on the record that the term "camera 
sensor data" is to be interpreted such that it must include additional sensor data (pan, 
zoom, tilt, and/or focus information, as in Figure 1, as examples, orientation). This is an 
explicit disclaimer of claim scope that creates a prosecution history estoppel. 

The best-applied prior art is given as it would be applied to claim 1 . It is noted 
that the overall combination - with the limitation of training as defined in the instant 
specification in section 1 C (as stated by attorney as being critical and as being the 
definition of training) - does not teaching training and/or a view modeler, where 
Kreitman is the closest available prior art. Therefore, this rejection was not used. 

A method of placing content into an image sequence, comprising: (Preamble is not 
given patentable weight, since it only recites a summary of the claim and/or an intended 
use, and the process steps and/or apparatus components are capable of standing on 
their own; see Rowe v. Dror, 112 F.3d 473, 42 USPQ2d 1550 (Fed. Cir. 1997), Pitney 
Bowes, Inc. v. Hewlett-Packard Co., 182 F.3d 1298, 1305, 51 USPQ2d 1161, 1165 
(Fed. Cir. 1999), and the like.) 

-Applying a transform function to visual content to be inserted into an image sequence; 
and (Moengen 1 :40-2:2, the idea of applying a "comet tail" to an ice hockey puck. Note 
further the Abstract, where the concept of a synthetic track applied to an image is 
stated. This tracking is done specifically by determining the location of an object using 
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electromagnetic energy or the like (see the Foxtrax system, as discussed in 1:40- 
2:2)(specifically, note 3:35-4:64). Clearly, applying a "comet tail" or synthetic track 
would comprise "applying a transform function to visual content to be inserted into an 
image sequence, e.g. a television picture.")(Kreitman describes applying a transform 
function to visual content to be inserted into an image sequence (see Kreitman, Figs. 2, 
4, and 7:64-8:9, wherein image 42 of Fig. 4 is transformed to match the perspective of 
the input video frame of Fig. 2 and is inserted into the video frame of Fig. 2))(Honey's 
patent is the FoxTrax system described in Moengen above. Honey tracks the location 
of the object and shows it with a comet tail on the screen as described above. Now, the 
system of Honey does apply a transform function to visual content, since the system 
automatically controls the camera so that it follows the object using electromagnetic 
energy emitted by the puck. Therefore, the camera is commanded to pan, scan, and 
rotate based on the determined position of the object. See Figure 7, where the camera 
has a PTZ (pan, tilt, and zoom) encoder that codes such information into the television 
signal. Furthermore, Honey clearly describes how the puck can be outlined or replaced 
with a different symbol. This is done entirely based (8:35-9:35) on the position of the 
puck and knowledge of the tilt, angle, etc of the camera based on the above sequence.) 
-Blending the transformed visual content with the image sequence; (Moengen Abstract 
states that the synthetic object overlays the movable object and represents it in the 
television picture (e.g. like the "comet tail" used by Foxtrax and the like))(Kreitman 
clearly teaches blending the transformed visual content with the image sequence (see 
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Kreitman 6:26-37, Fig. 4C, and 8:1-9, wherein transformed blending mask 58 is utilized 
to blend the image 42 with the colors of the field on which the image 42 is implanted)) 
-Wherein the applying step and the blending step together result in insertion of the 
content into the image sequence such that the content appears at a target location as if 
it had been part of the original scene displayed by the image sequence; and (Kreitman 
8:1-9, wherein if players walk on the part of the court 32 where the image 42 is 
implanted, they will appear to walk "over" the implanted image); 
-Wherein the applying step comprises all geometric transformations of the visual 
content, is performed without reference to any content contained in the image 
sequence, is performed with reference to a model that resulted from training, and is 
performed without reference to any three-dimensional real world space locations or 
dimensions. (Kreitman, col. 7, line 64 to col. 8, line 6, wherein the transforming step 
performed by transformer 64 comprises all applied geometric transformations of the 
visual content)(Kreitman clearly utilizes a model as defined above; see 3:10-15 as an 
example. )(Moengen clearly does not utilize any three-dimensional real world space 
locations as defined by applicant's specification, e.g. with respect to the camera - such 
locations are merely defined with respect to the model of the scene (as in Kreitman)). 

Moengen (and Honey) teach a system where various types of system track the 
location of various objects within a scene. These objects are tracking using non-visible 
energy (e.g. radar, infrared, or other types of electromagnetic energy in the case of 
honey). These various objects can be players or pucks or balls (1 1 :8-44) as well as 
(15:24-60) objects within a race or other moving targets. The only requirement is that 
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such items have transponders on them. Now, Honey further teaches that the camera 
information (9:30-35) can be used along with the location information from the tracking 
sensors to find the location of the puck within a frame. Specifically, with multiple 
cameras and sensors, the location of puck is found using the invisible radiation method 
(e.g. IR or radar). Now, this can be done with various representations of the scene, 
such as taught by Moengen in (8:63-9:15, 12:32-13:35,13:55-14:55) and the like. 
Obviously, if the object is not visible but is obscured by something, the system would 
need to know where to insert the synthetic object, which would therefore require a 
model, which is suggested in (14:65-15:20) that representations of a physical space 
could be used, which clearly suggests a model. 

However, Moengen (and Honey) do not expressly teach the use of a model as 
stated in the above claim. Kreitman, an analogous art, clearly teaches the use of a 
model generated before the actual event (as in by training or calibration). Now, that 
reference admittedly does not teach certain details of the claims (as noted in applicant's 
Remarks). However, Moengen clearly does at least contemplate the use of a model, 
and if Moengen overlays a track in a scene where the object is obscured by other items, 
it must prima facie be capable of utilizing a model. 

Kreitman, as noted previously, also teaches that the object can be placed in any 
location on the model and that a mask is used to determine what part of it, if any, is 
visible. However, in place of using the visual method of determining whether or not an 
object is in the way of the model, the methods of Moengen and Honey could clearly be 
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used to determine the location of an object, such that no real-world coordinate data 
would be necessary to perform the steps of Kreitman. 

Now, Kreitman does teach the use of a model created by a calibration or training 
process, where a model of the scene is created. Note that the instant claim does not 
state the model cannot be created using the camera or any other technique, only that 
the insertion uses a model created by training.' Therefore, the teachings of Kreitman on 
how to create a model are quite relevant herein. 

Therefore, since Moengen clearly contemplates having various objects tracked in 
a scene, where the objects could obscure each other and/or be obscured by objects in 
the scene, it would be obvious that utilizing a model to avoid occlusion problems 
between tracks or the like would be advantageous, as noted in Kreitman (1 :60-2:45), 
where the utility of having a model and masking is pointed out. Obviously, the model of 
Kreitman would serve therein, along with the idea of masking the object shown in the 
model if it is occluded by another object, e.g. one of the transponders being tracked in 
Moengen / Honey. 

Motivation for combination of Moengen and Honey is simply that Moengen cites 
Honey as relevant prior art and discusses various aspects of it. Also, Moengen is silent 
on whether or not the cameras transmit their PTZ data as set forth in Honey, which is 
necessary for the system to operate correctly without the use of coordinate information. 



Conclusion 
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The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. See 892 for citation. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Eric Woods whose telephone number is 571-272-7775. 
The examiner can normally be reached on M-F 7:30-5:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Michael Razavi can be reached on 571-272-7664. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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ATTORNEY DOCKET NO.: 103666.00101 PATENT 

PROPOSED AMENDMENTS 
1 . (Currently Amended) A method of placing content into an image sequence, comprising: 
applying a transfonn function to visual content to be inserted into an image sequence; and 
blending the transformed visual content with the image sequence; 

wherein the applying step and the blending step together result in insertion of the content 
into the image sequence such that the content appears at a target location as if it had been part of 
the original scene displayed by the image sequence; and 

wherein the applying step comprises all applied geometric transformations of the visual 
content, is performed without reference to any content contained in the image sequence* is 
performed with reference to a view model that resulted beforehand from training that used 
camera sensor data , and is performed without reference to any three dimensional real world 
space locations or dimensions. 

18. (Currently Amended) A method of placing a virtual marker line into an image sequence, 
comprising: 

applying a transform function to a marker line that is to be inserted into an image 
sequence that was produced by a camera; and 

blending the transformed line with the image sequence; 

wherein the applying step and the blending step together result in insertion of the line into 
the image sequence such that the line appears at a target location as if it had been part of the 
original scene displayed by the image sequence; and 

wherein the applying step comprises all applied geometric transformations of the line, is 
performed without reference to any real-time content contained in the image sequence, is 
performed with reference to a view model that was defined beforehand in a setup phase feat 

PT: #248426 vl (5B_Q0J !.DOC) 
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includMtraining using camera sensor data, and is performed without reference to any location of 
the camera. 

22. (Currently Amended) A method of placing a virtual sign into an image sequence, 
comprising: 

applying a transform function to a virtual sign that is to be placed into an image sequence 
that was produced by a camera; and 

blending the transformed sign with the image sequence; 

wherein the applying step and the blending step together result in insertion of the sign 
into the image sequence such that the sign appears at a target location as if it had been part of the 
original scene displayed by the image sequence; and 

wherein the applying step comprises all applied geometric transformations of the sign, is 
performed without reference to any content contained in the image sequence, is performed with 
reference to a view model that was defined beforehand in a setup phase that included framing 
using camera sensor data, and is performed without reference to location of the camera. 

26, (Currently Amended) A method of placing content on a moving object in an image 
sequence, comprising: 

applying a transform function to visual content to be inserted into an image sequence, 
wherein the image sequence includes a moving object; and 

blending the transformed visual content with the image sequence; 

wherein the applying step and the blending step together result in insertion of the content 
into the image sequence such that the content appears at a target location as if it had been part of 

"2- 
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the original scene displayed by the image sequence, and such that the content is located on the 
moving object as the object moves in the scene; and 

wherein the applying step comprises all applied geometric transformations of the visual 
content, is performed without reference to any content contained in the ixt^age sequence, is 
performed with reference to a view model that was defined beforehand in a setup phase that 
included training using camera sensor data, and is performed without reference to any three 
dimensional real world space locations or dimensions. 

i 

30. (Currently Amended) A method of inserting dynamically changing visual content into an 
image sequence, comprising: 

applying a transform iunction to dynamically changing visual content; and 

blending the transformed dynamically changing visual content with an image sequence; 

wherein the applying step and the blending step together result in insertion of the content 
into the image sequence such that the dynamically changing visual content appears as if it had 
been part of the original scene displayed by the image sequence; 

wherein said applying step comprises all applied geometric transformations of the 
dynamically changing visual content, is performed without reference to any real-time image 
content contained in the image sequence, is performed with reference to a jjjgjy model that 
resulted beforehand from training that used camera sensor data, and is performed without 
reference to any three dimensional real world space locations as related to the image sequence. 
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